Motorized full field adaptive microscope

ABSTRACT

An optical imaging system for imaging a target during a medical procedure. The imaging system includes an optical assembly including moveable zoom optics and moveable focus optics. The system includes a zoom actuator and a focus actuator for positioning the zoom and focus optics, respectively. The system includes a controller for controlling the zoom and focus actuators independently in response to received control input. The system includes a camera for capturing an image of the target from the optical assembly. The system may be capable of performing autofocus during a medical procedure.

FIELD

The present disclosure is generally related to optical imaging systems,including optical imaging systems suitable for use in image guidedmedical procedures.

BACKGROUND

Surgical microscopes are often used during surgical procedures toprovide a detailed or magnified view of the surgical site. In somecases, separate narrow field and wide field scopes may be used withinthe same surgical procedure to obtain image views with different zoomranges.

Often, adjusting the zoom and focus of such a surgical microscoperequire the user (e.g., a surgeon) to manually adjust the optics of themicroscope, which may be difficult, time-consuming and frustrating,particularly during a surgical procedure.

As well, image capture cameras and light sources often are separatepieces of equipment from the surgical microscope, such that the specificcamera and light source used with a given surgical microscope may bedifferent for different medical centers and even for different surgicalprocedures within the same medical center. This may result in aninconsistency in the images obtained, which may make it difficult orimpossible to compare images between different medical centers.

SUMMARY

In some examples, the present disclosure provides an optical imagingsystem for imaging a target during a medical procedure. The systemincludes: an optical assembly including moveable zoom optics andmoveable focus optics; a zoom actuator for positioning the zoom optics;a focus actuator for positioning the focus optics; a controller forcontrolling the zoom actuator and the focus actuator in response toreceived control input; and a camera for capturing an image of thetarget from the optical assembly; wherein the zoom optics and the focusoptics are independently moveable by the controller using the zoomactuator and the focus actuator, respectively; and wherein the opticalimaging system is configured to operate at a minimum working distancefrom the target, the working distance being defined between an apertureof the optical assembly and the target.

In some examples, the present disclosure provides a processor forcontrolling the optical imaging system disclosed herein. The processoris configured to: provide a user interface to receive control input, viaan input device coupled to the processor, for controlling the zoomactuator and the focus actuator; transmit control instructions to thecontroller of the optical imaging system to adjust zoom and focus inaccordance with the control input; and receive image data from thecamera for outputting to an output device coupled to the processor.

In some examples, the present disclosure provides a system for opticalimaging during a medical procedure. The system includes: the opticalimaging system disclosed herein; a positioning system for positioningthe optical imaging system; and a navigation system for tracking each ofthe optical imaging system and the positioning system relative to thetarget.

In some examples, the present disclosure provides a method ofautofocusing using an optical imaging system during a medical procedure,the optical imaging system including motorized focus optics and acontroller for positioning the focus optics. The method includes:determining a working distance between an imaging target and an apertureof the optical imaging system; determining a desired position of thefocus optics based on the working distance; and positioning the focusoptics at the desired position.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made, by way of example, to the accompanyingdrawings which show example embodiments of the present application, andin which:

FIG. 1 illustrates the insertion of an access port into a human brain,for providing access to internal brain tissue during an example medicalprocedure;

FIG. 2A shows an example navigation system to support image guidedsurgery;

FIG. 2B is a diagram illustrating system components of an examplenavigation system;

FIG. 3 is a block diagram illustrating an example control and processingsystem that may be used in the example navigation systems of FIGS. 2Aand 2B;

FIG. 4A is a flow chart illustrating an example method involved in asurgical procedure that may be implemented using the example navigationsystems of FIGS. 2A and 2B;

FIG. 4B is a flow chart illustrating an example method of registering apatient for a surgical procedure as outlined in FIG. 4A;

FIG. 5 shows the use of an example optical imaging system during amedical procedure;

FIG. 6 is a block diagram of an example optical imaging system;

FIGS. 7 and 8 are different perspective views of an example opticalimaging system;

FIG. 9 is a flowchart illustrating an example method of autofocusingusing an example optical imaging system;

FIG. 10 is a flowchart illustrating an example method of autofocusingrelative to a medical instrument, using an example optical imagingsystem; and

FIG. 11 illustrates an example method of autofocusing relative to amedical instrument, using an example optical imaging system.

Similar reference numerals may have been used in different figures todenote similar components.

DESCRIPTION OF EXAMPLE EMBODIMENTS

The systems and methods described herein may be useful in the field ofneurosurgery, including oncological care, neurodegenerative disease,stroke, brain trauma and orthopedic surgery. The teachings of thepresent disclosure may be applicable to other conditions or fields ofmedicine. It should be noted that while the present disclosure describesexamples in the context of neurosurgery, the present disclosure may beapplicable to other surgical procedures that may use intraoperativeoptical imaging.

Various example apparatuses or processes will be described below. Noexample embodiment described below limits any claimed embodiment and anyclaimed embodiments may cover processes or apparatuses that differ fromthose examples described below. The claimed embodiments are not limitedto apparatuses or processes having all of the features of any oneapparatus or process described below or to features common to multipleor all of the apparatuses or processes described below. It is possiblethat an apparatus or process described below is not part of any claimedembodiment.

Furthermore, numerous specific details are set forth in order to providea thorough understanding of the disclosure. However, it will beunderstood by those of ordinary skill in the art that the embodimentsdescribed herein may be practiced without these specific details. Inother instances, well-known methods, procedures and components have notbeen described in detail so as not to obscure the embodiments describedherein.

As used herein, the terms, “comprises” and “comprising” are to beconstrued as being inclusive and open ended, and not exclusive.Specifically, when used in the specification and claims, the terms,“comprises” and “comprising” and variations thereof mean the specifiedfeatures, steps or components are included. These terms are not to beinterpreted to exclude the presence of other features, steps orcomponents.

As used herein, the term “exemplary” or “example” means “serving as anexample, instance, or illustration,” and should not be construed aspreferred or advantageous over other configurations disclosed herein.

As used herein, the terms “about”, “approximately”, and “substantially”are meant to cover variations that may exist in the upper and lowerlimits of the ranges of values, such as variations in properties,parameters, and dimensions. In one non-limiting example, the terms“about”, “approximately”, and “substantially” may be understood to meanplus or minus 10 percent or less.

Unless defined otherwise, all technical and scientific terms used hereinare intended to have the same meaning as commonly understood by one ofordinary skill in the art. Unless otherwise indicated, such as throughcontext, as used herein, the following terms are intended to have thefollowing meanings:

As used herein, the phrase “access port” refers to a cannula, conduit,sheath, port, tube, or other structure that is insertable into asubject, in order to provide access to internal tissue, organs, or otherbiological substances. In some embodiments, an access port may directlyexpose internal tissue, for example, via an opening or aperture at adistal end thereof, and/or via an opening or aperture at an intermediatelocation along a length thereof. In other embodiments, an access portmay provide indirect access, via one or more surfaces that aretransparent, or partially transparent, to one or more forms of energy orradiation, such as, but not limited to, electromagnetic waves andacoustic waves.

As used herein the phrase “intraoperative” refers to an action, process,method, event or step that occurs or is carried out during at least aportion of a medical procedure. Intraoperative, as defined herein, isnot limited to surgical procedures, and may refer to other types ofmedical procedures, such as diagnostic and therapeutic procedures.

Some embodiments of the present disclosure relate to minimally invasivemedical procedures that are performed via an access port, wherebysurgery, diagnostic imaging, therapy, or other medical procedures (e.g.minimally invasive medical procedures) are performed based on access tointernal tissue through the access port.

In the example of a port-based surgery, a surgeon or robotic surgicalsystem may perform a surgical procedure involving tumor resection inwhich the residual tumor remaining after is minimized, while alsominimizing the trauma to the intact white and grey matter of the brain.In such procedures, trauma may occur, for example, due to contact withthe access port, stress to the brain matter, unintentional impact withsurgical devices, and/or accidental resection of healthy tissue. A keyto minimizing trauma is ensuring that the surgeon performing theprocedure has the best possible view of the surgical site of interestwithout having to spend excessive amounts of time and concentrationrepositioning tools, scopes and/or cameras during the medical procedure.

FIG. 1 illustrates the insertion of an access port into a human brain,for providing access to internal brain tissue during a medicalprocedure. In FIG. 1, an access port 12 is inserted into a human brain10, providing access to internal brain tissue. The access port 12 mayinclude such instruments as catheters, surgical probes, or cylindricalports such as the NICO BrainPath™ Surgical tools and instruments maythen be inserted within the lumen of the access port 12 in order toperform surgical, diagnostic or therapeutic procedures, such asresecting tumors as necessary. In the example of a port-based surgery, astraight or linear access port 12 is typically guided down a sulci pathof the brain. Surgical instruments would then be inserted down theaccess port 12.

The present disclosure applies equally well to catheters, DBS needles, abiopsy procedure, and also to biopsies and/or catheters in other medicalprocedures performed on other parts of the body, as well as to medicalprocedures that do not use an access port. Various examples of thepresent disclosure may be generally suitable for use in any medicalprocedure that may use optical imaging systems.

In FIG. 2A, an exemplary navigation system environment 200 is shown,which may be used to support navigated image-guided surgery. As shown inFIG. 2, surgeon 201 conducts a surgery on a patient 202 in an operatingroom (OR) environment. A medical navigation system 205 may include anequipment tower, tracking system, displays and tracked instruments toassist the surgeon 201 during his procedure. An operator 203 may also bepresent to operate, control and provide assistance for the medicalnavigation system 205.

FIG. 2B shows a diagram illustrating an example medical navigationsystem 205 in greater detail. The disclosed optical imaging system maybe used in the context of the medical navigation system 205. The medicalnavigation system 205 may include one or more displays 206, 211 fordisplaying a video image, an equipment tower 207, and a positioningsystem 208, such as a mechanical arm, which may support an opticalimaging system 500 (which may include an optical scope). One or more ofthe displays 206, 211 may include a touch-sensitive display forreceiving touch input. The equipment tower 207 may be mounted on a frame(e.g., a rack or cart) and may contain a power supply and a computer orcontroller that may execute planning software, navigation softwareand/or other software to manage the positioning system 208 one or moreinstruments tracked by the navigation system 205. In some examples, theequipment tower 207 may be a single tower configuration operating withdual displays 206, 211, however other configurations may also exist(e.g., dual tower, single display, etc.). Furthermore, the equipmenttower 207 may also be configured with a universal power supply (UPS) toprovide for emergency power, in addition to a regular AC adapter powersupply.

A portion of the patient's anatomy may be held in place by a holder. Forexample, as shown the patient's head and brain may be held in place by ahead holder 217. An access port 12 and associated introducer 210 may beinserted into the head, to provide access to a surgical site in thehead. The imaging system 500 may be used to view down the access port 12at a sufficient magnification to allow for enhanced visibility down theaccess port 12. The output of the imaging system 500 may be received byone or more computers or controllers to generate a view that may bedepicted on a visual display (e.g., one or more displays 206, 211).

In some examples, the navigation system 205 may include a trackedpointer 222. The tracked pointer 222, which may include markers 212 toenable tracking by a tracking camera 213, may be used to identify points(e.g., fiducial points) on a patient. An operator, typically a nurse orthe surgeon 201, may use the tracked pointer 222 to identify thelocation of points on the patient 202, in order to register the locationof selected points on the patient 202 in the navigation system 205. Itshould be noted that a guided robotic system with closed loop controlmay be used as a proxy for human interaction. Guidance to the roboticsystem may be provided by any combination of input sources such as imageanalysis, tracking of objects in the operating room using markers placedon various objects of interest, or any other suitable robotic systemguidance techniques.

Fiducial markers 212 may be connected to the introducer 210 for trackingby the tracking camera 213, which may provide positional information ofthe introducer 210 from the navigation system 205. In some examples, thefiducial markers 212 may be alternatively or additionally attached tothe access port 12. In some examples, the tracking camera 213 may be a3D infrared optical tracking stereo camera similar to one made byNorthern Digital Imaging (NDI). In some examples, the tracking camera213 may be instead an electromagnetic system (not shown), such as afield transmitter that may use one or more receiver coils located on thetool(s) to be tracked. A known profile of the electromagnetic field andknown position of receiver coil(s) relative to each other may be used toinfer the location of the tracked tool(s) using the induced signals andtheir phases in each of the receiver coils. Operation and examples ofthis technology is further explained in Chapter 2 of “Image-GuidedInterventions Technology and Application,” Peters, T.; Cleary, K., 2008,ISBN: 978-0-387-72856-7, incorporated herein by reference. Location dataof the positioning system 208 and/or access port 12 may be determined bythe tracking camera 213 by detection of the fiducial markers 212 placedon or otherwise in fixed relation (e.g., in rigid connection) to any ofthe positioning system 208, the access port 12, the introducer 210, thetracked pointer 222 and/or other tracked instruments. The fiducialmarker(s) 212 may be active or passive markers. A display 206, 2011 mayprovide an output of the computed data of the navigation system 205. Insome examples, the output provided by the display 206, 211 may includeaxial, sagittal and coronal views of patient anatomy as part of amulti-view output.

The active or passive fiducial markers 212 may be placed on tools (e.g.,the access port 12 and/or the imaging system 500) to be tracked, todetermine the location and orientation of these tools using the trackingcamera 213 and navigation system 205. The markers 212 may be captured bya stereo camera of the tracking system to give identifiable points fortracking the tools. A tracked tool may be defined by a grouping ofmarkers 212, which may define a rigid body to the tracking system. Thismay in turn be used to determine the position and/or orientation in 3Dof a tracked tool in a virtual space. The position and orientation ofthe tracked tool in 3D may be tracked in six degrees of freedom (e.g.,x, y, z coordinates and pitch, yaw, roll rotations), in five degrees offreedom (e.g., x, y, z, coordinate and two degrees of free rotation),but preferably tracked in at least three degrees of freedom (e.g.,tracking the position of the tip of a tool in at least x, y, zcoordinates). In typical use with navigation systems, at least threemarkers 212 are provided on a tracked tool to define the tool in virtualspace, however it is known to be advantageous for four or more markers212 to be used.

Camera images capturing the markers 212 may be logged and tracked, by,for example, a closed circuit television (CCTV) camera. The markers 212may be selected to enable or assist in segmentation in the capturedimages. For example, infrared (IR)-reflecting markers and an IR lightsource from the direction of the camera may be used. An example of suchan apparatus may be tracking devices such as the Polaris® systemavailable from Northern Digital Inc. In some examples, the spatialposition and orientation of the tracked tool and/or the actual anddesired position and orientation of the positioning system 208 may bedetermined by optical detection using a camera. The optical detectionmay be done using an optical camera, rendering the markers 212 opticallyvisible.

In some examples, the markers 212 (e.g., reflectospheres) may be used incombination with a suitable tracking system, to determine the spatialpositioning position of the tracked tools within the operating theatre.Different tools and/or targets may be provided with respect to sets ofmarkers 212 in different configurations. Differentiation of thedifferent tools and/or targets and their corresponding virtual volumesmay be possible based on the specification configuration and/ororientation of the different sets of markers 212 relative to oneanother, enabling each such tool and/or target to have a distinctindividual identity within the navigation system 205. The individualidentifiers may provide information to the system, such as informationrelating to the size and/or shape of the tool within the system. Theidentifier may also provide additional information such as the tool'scentral point or the tool's central axis, among other information. Thevirtual tool may also be determinable from a database of tools stored inor provided to the navigation system 205. The markers 212 may be trackedrelative to a reference point or reference object in the operating room,such as the patient 202.

Various types of markers may be used. The markers 212 may all be thesame type or may include a combination of two or more different types.Possible types of markers that could be used may include reflectivemarkers, radiofrequency (RF) markers, electromagnetic (EM) markers,pulsed or un-pulsed light-emitting diode (LED) markers, glass markers,reflective adhesives, or reflective unique structures or patterns, amongothers. RF and EM markers may have specific signatures for the specifictools they may be attached to. Reflective adhesives, structures andpatterns, glass markers, and LED markers may be detectable using opticaldetectors, while RF and EM markers may be detectable using antennas.Different marker types may be selected to suit different operatingconditions. For example, using EM and RF markers may enable tracking oftools without requiring a line-of-sight from a tracking camera to themarkers 212, and using an optical tracking system may avoid additionalnoise from electrical emission and detection systems.

In some examples, the markers 212 may include printed or 3D designs thatmay be used for detection by an auxiliary camera, such as a wide-fieldcamera (not shown) and/or the imaging system 500. Printed markers mayalso be used as a calibration pattern, for example to provide distanceinformation (e.g., 3D distance information) to an optical detector.Printed identification markers may include designs such as concentriccircles with different ring spacing and/or different types of bar codes,among other designs. In some examples, in addition to or in place ofusing markers 212, the contours of known objects (e.g., the side of theaccess port 12) could be captured by and identified using opticalimaging devices and the tracking system.

A guide clamp 218 (or more generally a guide) for holding the accessport 12 may be provided. The guide clamp 218 may allow the access port12 to be held at a fixed position and orientation while freeing up thesurgeon's hands. An articulated arm 219 may be provided to hold theguide clamp 218. The articulated arm 219 may have up to six degrees offreedom to position the guide clamp 218. The articulated arm 219 may belockable to fix its position and orientation, once a desired position isachieved. The articulated arm 219 may be attached or attachable to apoint based on the patient head holder 217, or another suitable point(e.g., on another patient support, such as on the surgical bed), toensure that when locked in place, the guide clamp 218 does not moverelative to the patient's head.

In a surgical operating room (or theatre), setup of a navigation systemmay be relatively complicated; there may be many pieces of equipmentassociated with the surgical procedure, as well as elements of thenavigation system 205. Further, setup time typically increases as moreequipment is added. To assist in addressing this, the navigation system205 may include two additional wide-field cameras to enable videooverlay information. Video overlay information can then be inserted intodisplayed images, such as images displayed on one or more of thedisplays 206, 211. The overlay information may illustrate the physicalspace where accuracy of the 3D tracking system (which is typically partof the navigation system) is greater, may illustrate the available rangeof motion of the positioning system 208 and/or the imaging system 500,and/or may help to guide head and/or patient positioning.

The navigation system 205 may provide tools to the neurosurgeon that mayhelp to provide more relevant information to the surgeon, and may assistin improving performance and accuracy of port-based neurosurgicaloperations. Although described in the present disclosure in the contextof port-based neurosurgery (e.g., for removal of brain tumors and/or fortreatment of intracranial hemorrhages (ICH)), the navigation system 205may also be suitable for one or more of: brain biopsy,functional/deep-brain stimulation, catheter/shunt placement (in thebrain or elsewhere), open craniotomies, and/orendonasal/skull-based/ear-nose-throat (ENT) procedures, among others.The same navigation system 205 may be used for carrying out any or allof these procedures, with or without modification as appropriate.

For example, although the present disclosure may discuss the navigationsystem 205 in the context of neurosurgery, the same navigation system205 may be used to carry out a diagnostic procedure, such as brainbiopsy. A brain biopsy may involve the insertion of a thin needle into apatient's brain for purposes of removing a sample of brain tissue. Thebrain tissue may be subsequently assessed by a pathologist to determineif it is cancerous, for example. Brain biopsy procedures may beconducted with or without a stereotactic frame. Both types of proceduresmay be performed using image-guidance. Frameless biopsies, inparticular, may be conducted using the navigation system 205.

In some examples, the tracking camera 213 may be part of any suitabletracking system. In some examples, the tracking camera 213 (and anyassociated tracking system that uses the tracking camera 213) may bereplaced with any suitable tracking system which may or may not usecamera-based tracking techniques. For example, a tracking system thatdoes not use the tracking camera 213, such as a radiofrequency trackingsystem, may be used with the navigation system 205.

FIG. 3 is a block diagram illustrating a control and processing system300 that may be used in the medical navigation system 205 shown in FIG.2B (e.g., as part of the equipment tower 207). As shown in FIG. 3, inone example, control and processing system 300 may include one or moreprocessors 302, a memory 304, a system bus 306, one or more input/outputinterfaces 308, a communications interface 310, and storage device 312.The control and processing system 300 may be interfaced with otherexternal devices, such as a tracking system 321, data storage 342, andexternal user input and output devices 344, which may include, forexample, one or more of a display, keyboard, mouse, sensors attached tomedical equipment, foot pedal, and microphone and speaker. Data storage342 may be any suitable data storage device, such as a local or remotecomputing device (e.g. a computer, hard drive, digital media device, orserver) having a database stored thereon. In the example shown in FIG.3, data storage device 342 includes identification data 350 foridentifying one or more medical instruments 360 and configuration data352 that associates customized configuration parameters with one or moremedical instruments 360. The data storage device 342 may also includepreoperative image data 354 and/or medical procedure planning data 356.Although the data storage device 342 is shown as a single device in FIG.3, it will be understood that in other embodiments, the data storagedevice 342 may be provided as multiple storage devices.

The medical instruments 360 may be identifiable by the control andprocessing unit 300. The medical instruments 360 may be connected to andcontrolled by the control and processing unit 300, or the medicalinstruments 360 may be operated or otherwise employed independent of thecontrol and processing unit 300. The tracking system 321 may be employedto track one or more medical instruments 360 and spatially register theone or more tracked medical instruments to an intraoperative referenceframe. For example, the medical instruments 360 may include trackingmarkers such as tracking spheres that may be recognizable by thetracking camera 213. In one example, the tracking camera 213 may be aninfrared (IR) tracking camera. In another example, as sheath placed overa medical instrument 360 may be connected to and controlled by thecontrol and processing unit 300.

The control and processing unit 300 may also interface with a number ofconfigurable devices, and may intraoperatively reconfigure one or moreof such devices based on configuration parameters obtained fromconfiguration data 352. Examples of devices 320, as shown in FIG. 3,include one or more external imaging devices 322, one or moreillumination devices 324, the positioning system 208, the trackingcamera 213, one or more projection devices 328, and one or more displays206, 211.

Exemplary aspects of the disclosure can be implemented via theprocessor(s) 302 and/or memory 304. For example, the functionalitiesdescribed herein can be partially implemented via hardware logic in theprocessor 302 and partially using the instructions stored in the memory304, as one or more processing modules or engines 370. Exampleprocessing modules include, but are not limited to, a user interfaceengine 372, a tracking module 374, a motor controller 376, an imageprocessing engine 378, an image registration engine 380, a procedureplanning engine 382, a navigation engine 384, and a context analysismodule 386. While the example processing modules are shown separately inFIG. 3, in some examples the processing modules 370 may be stored in thememory 304 and the processing modules 370 may be collectively referredto as processing modules 370. In some examples, two or more modules 370may be used together to perform a function. Although depicted asseparate modules 370, the modules 370 may be embodied as a unified setof computer-readable instructions (e.g., stored in the memory 304)rather than distinct sets of instructions.

It is to be understood that the system is not intended to be limited tothe components shown in FIG. 3. One or more components of the controland processing system 300 may be provided as an external component ordevice. In one example, the navigation module 384 may be provided as anexternal navigation system that is integrated with the control andprocessing system 300.

Some embodiments may be implemented using the processor 302 withoutadditional instructions stored in memory 304. Some embodiments may beimplemented using the instructions stored in memory 304 for execution byone or more general purpose microprocessors. Thus, the disclosure is notlimited to a specific configuration of hardware and/or software.

In some examples, the navigation system 205, which may include thecontrol and processing unit 300, may provide tools to the surgeon thatmay help to improve the performance of the medical procedure and/orpost-operative outcomes. In addition to removal of brain tumours andintracranial hemorrhages (ICH), the navigation system 205 can also beapplied to a brain biopsy, a functional/deep-brain stimulation, acatheter/shunt placement procedure, open craniotomies,endonasal/skull-based/ENT, spine procedures, and other parts of the bodysuch as breast biopsies, liver biopsies, etc. While several exampleshave been provided, examples of the present disclosure may be applied toany suitable medical procedure.

FIG. 4A is a flow chart illustrating an example method 400 of performinga port-based surgical procedure using a navigation system, such as themedical navigation system 205 described in relation to FIGS. 2A and 2B.At a first block 402, the port-based surgical plan is imported.

Once the plan has been imported into the navigation system at the block402, the patient is affixed into position using a body holdingmechanism. The head position is also confirmed with the patient plan inthe navigation system (block 404), which in one example may beimplemented by the computer or controller forming part of the equipmenttower 207.

Next, registration of the patient is initiated (block 406). The phrase“registration” or “image registration” refers to the process oftransforming different sets of data into one coordinate system. Data mayinclude multiple photographs, data from different sensors, times,depths, or viewpoints. The process of “registration” is used in thepresent application for medical imaging in which images from differentimaging modalities are co-registered. Registration is used in order tobe able to compare or integrate the data obtained from these differentmodalities.

Those skilled in the relevant arts will appreciate that there arenumerous registration techniques available and one or more of thetechniques may be applied to the present example. Non-limiting examplesinclude intensity-based methods that compare intensity patterns inimages via correlation metrics, while feature-based methods findcorrespondence between image features such as points, lines, andcontours. Image registration methods may also be classified according tothe transformation models they use to relate the target image space tothe reference image space. Another classification can be made betweensingle-modality and multi-modality methods. Single-modality methodstypically register images in the same modality acquired by the samescanner or sensor type, for example, a series of magnetic resonance (MR)images may be co-registered, while multi-modality registration methodsare used to register images acquired by different scanner or sensortypes, for example in magnetic resonance imaging (MRI) and positronemission tomography (PET). In the present disclosure, multi-modalityregistration methods may be used in medical imaging of the head and/orbrain as images of a subject are frequently obtained from differentscanners. Examples include registration of brain computerized tomography(CT)/MRI images or PET/CT images for tumor localization, registration ofcontrast-enhanced CT images against non-contrast-enhanced CT images, andregistration of ultrasound and CT.

FIG. 4B is a flow chart illustrating an example method involved inregistration block 406 as outlined in FIG. 4A, in greater detail. If theuse of fiducial touch points (440) is contemplated, the method involvesfirst identifying fiducials on images (block 442), then touching thetouch points with a tracked instrument (block 444). Next, the navigationsystem computes the registration to reference markers (block 446).

Alternately, registration can also be completed by conducting a surfacescan procedure (block 450). The block 450 is presented to show analternative approach, but may not typically be used when using afiducial pointer. First, the face is scanned using a 3D scanner (block452). Next, the face surface is extracted from MR/CT data (block 454).Finally, surfaces are matched to determine registration data points(block 456).

Upon completion of either the fiducial touch points (440) or surfacescan (450) procedures, the data extracted is computed and used toconfirm registration at block 408, shown in FIG. 4A.

Referring back to FIG. 4A, once registration is confirmed (block 408),the patient is draped (block 410). Typically, draping involves coveringthe patient and surrounding areas with a sterile barrier to create andmaintain a sterile field during the surgical procedure. The purpose ofdraping is to eliminate the passage of microorganisms (e.g., bacteria)between non-sterile and sterile areas. At this point, conventionalnavigation systems require that the non-sterile patient reference isreplaced with a sterile patient reference of identical geometry locationand orientation.

Upon completion of draping (block 410), the patient engagement pointsare confirmed (block 412) and then the craniotomy is prepared andplanned (block 414).

Upon completion of the preparation and planning of the craniotomy (block414), the craniotomy is cut and a bone flap is temporarily removed fromthe skull to access the brain (block 416). Registration data is updatedwith the navigation system at this point (block 422).

Next, the engagement within craniotomy and the motion range areconfirmed (block 418). Next, the procedure advances to cutting the duraat the engagement points and identifying the sulcus (block 420).

Thereafter, the cannulation process is initiated (block 424).Cannulation involves inserting a port into the brain, typically along asulci path as identified at 420, along a trajectory plan. Cannulation istypically an iterative process that involves repeating the steps ofaligning the port on engagement and setting the planned trajectory(block 432) and then cannulating to the target depth (block 434) untilthe complete trajectory plan is executed (block 424).

Once cannulation is complete, the surgeon then performs resection (block426) to remove part of the brain and/or tumor of interest. The surgeonthen decannulates (block 428) by removing the port and any trackinginstruments from the brain. Finally, the surgeon closes the dura andcompletes the craniotomy (block 430). Some aspects of FIG. 4A arespecific to port-based surgery, such as portions of blocks 428, 420, and434, but the appropriate portions of these blocks may be skipped orsuitably modified when performing non-port based surgery.

When performing a surgical procedure using a medical navigation system205, as outlined in connection with FIGS. 4A and 4B, the medicalnavigation system 205 may acquire and maintain a reference of thelocation of the tools in use as well as the patient in three dimensional(3D) space. In other words, during a navigated neurosurgery, there maybe a tracked reference frame that is fixed (e.g., relative to thepatient's skull). During the registration phase of a navigatedneurosurgery (e.g., the step 406 shown in FIGS. 4A and 4B), atransformation is calculated that maps the frame of reference ofpreoperative MRI or CT imagery to the physical space of the surgery,specifically the patient's head. This may be accomplished by thenavigation system 205 tracking locations of fiducial markers fixed tothe patient's head, relative to the static patient reference frame. Thepatient reference frame is typically rigidly attached to the headfixation device, such as a Mayfield clamp. Registration is typicallyperformed before the sterile field has been established (e.g., the step410 shown in FIG. 4A).

FIG. 5 illustrates use of an example imaging system 500, describedfurther below, in a medical procedure. Although FIG. 5 shows the imagingsystem 500 being used in the context of a navigation system environment200 (e.g., using a navigation system as described above), the imagingsystem 500 may also be used outside of a navigation system environment(e.g., without any navigation support).

An operator, typically a surgeon 201, may use the imaging system 500 toobserve the surgical site (e.g., to look down an access port). Theimaging system 500 may be attached to a positioning system 208 (e.g., acontrollable and adjustable robotic arm). The position and orientationof the positioning system 208, imaging system 500 and/or access port maybe tracked using a tracking system, such as described for the navigationsystem 205. The distance d between the imaging system 500 (morespecifically, the aperture of the imaging system 500) and the viewingtarget (e.g., the surface of the surgical site) may be referred to asthe working distance. The imaging system 500 may be designed to be usedin a predefined range of working distance (e.g., in the range of about15 cm to about 75 cm). It should be noted that, if the imaging system500 is mounted on the positioning system 208, the actual available rangeof working distance may be dependent on both the working distance of theimaging system 500 as well as the workspace and kinematics of thepositioning system 208.

FIG. 6 is a block diagram showing components of an example imagingsystem 500. The imaging system 500 may include an optical assembly 505(also referred to as an optical train). The optical assembly 505 mayinclude optics (e.g., lenses, optical fibers, etc.) for focusing andzooming on the viewing target. The optical assembly 505 may include zoomoptics 510 (which may include one or more zoom lenses) and focus optics515 (which may include one or more focus lenses). Each of the zoomoptics 510 and focus optics 515 are independently moveable within theoptical assembly, in order to adjust the zoom and focus, respectively.Where the zoom optics 510 and/or the focus optics 515 include more thanone lens, each individual lens may be independently moveable. Theoptical assembly 505 may include an aperture (not shown), which may beadjustable.

The imaging system 500 may include a zoom actuator 520 and a focusactuator 525 for positioning the zoom optics 510 and the focus optics515, respectively. The zoom actuator 520 and/or the focus actuator 525may be an electric motor, or other types of actuators including, forexample, pneumatic actuators, hydraulic actuators, shape-changingmaterials (e.g., piezoelectric materials or other smart materials) orengines, among other possibilities. Although the term “motorized” isused in the present disclosure, it should be understood that the use ofthis term does not limit the present disclosure to use of motorsnecessarily, but is intended to cover all suitable actuators, includingmotors. Although the zoom actuator 520 and the focus actuator 525 areshown outside of the optical assembly 505, in some examples the zoomactuator 520 and the focus actuator 525 may be part of or integratedwith the optical assembly 505. The zoom actuator 520 and the focusactuator 525 may operate independently, to control positioning of thezoom optics 510 and the focus optics 515, respectively. The lens(es) ofthe zoom optics 510 and/or the focus optics 515 may be each mounted on alinear stage (e.g., a motion system that restricts an object to move ina single axis, which may include a linear guide and an actuator; or aconveyor system such as a conveyor belt mechanism) that is moved by thezoom actuator 520 and/or the focus actuator 525, respectively, tocontrol positioning of the zoom optics 510 and/or the focus optics 515.In some examples, the zoom optics 510 may be mounted on a linear stagethat is driven, via a belt drive, by the zoom actuator 520, while thefocus optics 515 is geared to the focus actuator 525. The independentoperation of the zoom actuator 520 and the focus actuator 525 may enablethe zoom and focus to be adjusted independently. Thus, when an image isin focus, the zoom may be adjusted without requiring further adjustmentsto the focus optics 515 to produce a focused image.

Operation of the zoom actuator 520 and the focus actuator 525 may becontrolled by a controller 530 (e.g., a microprocessor) of the imagingsystem 500. The controller 530 may receive control input (e.g., from anexternal system, such as an external processor or an input device). Thecontrol input may indicate a desired zoom and/or focus, and thecontroller 530 may in response cause the zoom actuator 520 and/of focusactuator 525 to move the zoom optics 510 and/or the focus optics 515accordingly to achieve the desired zoom and/or focus. In some examples,the zoom optics 510 and/or the focus optics 515 may be moved or actuatedwithout the use of the zoom actuator 520 and/or the focus actuator 525.For example, the focus optics 515 may use electrically-tunable lenses orother deformable material that may be controlled directly by thecontroller 530.

By providing the controller 530, the zoom actuator 520 and the focusactuator 525 all as part of the imaging system 500, the imaging system500 may enable an operator (e.g., a surgeon) to control zoom and/orfocus during a medical procedure without having to manually adjust thezoom and/or focus optics 510, 515. For example, the operator may providecontrol input to the controller 530 verbally (e.g., via a voicerecognition input system), by instructing an assistant to enter controlinput into an external input device (e.g., into a user interfaceprovided by a workstation), using a foot pedal, or by other such means.In some examples, the controller 530 may carry out preset instructionsto maintain the zoom and/or focus at preset values (e.g., to performautofocusing) without requiring further control input during the medicalprocedure.

An external processor (e.g., a processor of a workstation or thenavigation system) in communication with the controller 530 may be usedto provide control input to the controller 530. For example, theexternal processor may provide a graphical user interface via which theoperator or an assistant may input instructions to control zoom and/orfocus of the imaging system 500. The controller 530 may alternatively oradditionally be in communication with an external input system (e.g., avoice recognition input system or a foot pedal).

The optical assembly 505 may also include one or more auxiliary optics540 (e.g., an adjustable aperture), which may be static or dynamic.Where the auxiliary optics 540 is dynamic, the auxiliary optics 540 maybe moved using an auxiliary actuator (not shown) which may be controlledby the controller 530.

The imaging system 500 may also include a camera 535 (e.g., ahigh-definition (HD) camera) that captures image data from the opticalassembly. Operation of the camera may be controlled by the controller530. The camera 535 may also output data to an external system (e.g., anexternal workstation or external output device) to view the capturedimage data. In some examples, the camera 535 may output data to thecontroller 530, which in turn transmits the data to an external systemfor viewing. By providing image data to an external system for viewing,the captured images may be viewed on a larger display and may bedisplayed together with other information relevant to the medicalprocedure (e.g., a wide-field view of the surgical site, navigationmarkers, 3D images, etc.). Providing the camera 535 with the imagingsystem 500 may help to improve the consistency of image quality amongdifferent medical centers.

Image data captured by the camera 535 may be displayed on a displaytogether with a wide-field view of the surgical site, for example in amultiple-view user interface. The portion of the surgical site that iscaptured by the camera 535 may be visually indicated in the wide-fieldview of the surgical site.

The imaging system 500 may include a three-dimensional (3D) scanner 545or 3D camera for obtaining 3D information of the viewing target. 3DInformation from the 3D scanner 545 may also be captured by the camera535, or may be captured by the 3D scanner 545 itself. Operation of the3D scanner 545 may be controlled by the controller 530, and the 3Dscanner 545 may transmit data to the controller 530. In some examples,the 3D scanner 545 may itself transmit data to an external system (e.g.,an external work station). 3D information from the 3D scanner 545 may beused to generate a 3D image of the viewing target (e.g., a 3D image of atarget tumor to be resected). 3D information may also be useful in anaugmented reality (AR) display provided by an external system. Forexample an AR display (e.g., provided via AR glasses) may, usinginformation from a navigation system to register 3D information withoptical images, overlay a 3D image of a target specimen on a real-timeoptical image (e.g., an optical image captured by the camera 535).

The controller 530 may be coupled to a memory 550. The memory 550 may beinternal or external of the imaging system 500. Data received by thecontroller 530 (e.g., image data from the camera 535 and/or 3D data fromthe 3D scanner) may be stored in the memory 550. The memory 550 may alsocontain instructions to enable the controller to operate the zoomactuator 520 and the focus actuator 525. For example, the memory 550 maystore instructions to enable the controller to perform autofocusing, asdiscussed further below.

The imaging system 500 may communicate with an external system (e.g., anavigation system or a workstation) via wired or wireless communication.In some examples, the imaging system 500 may include a wirelesstransceiver (not shown) to enable wireless communication.

In some examples, the imaging system 500 may include a power source(e.g., a battery) or a connector to a power source (e.g., an ACadaptor). In some examples, the imaging system 500 may receive power viaa connection to an external system (e.g., an external workstation orprocessor).

In some examples, the imaging system 500 may include a light source (notshown). In some examples, the light source may not itself generate lightbut rather direct light from another light generating component. Forexample, the light source may be an output of a fibre optics cableconnected to another light generating component, which may be part ofthe imaging system 500 or external to the imaging system 500. The lightsource may be mounted near the aperture of the optical assembly, todirect light to the viewing target. Providing the light source with theimaging system 500 may help to improve the consistency of image qualityamong different medical centers. In some examples, the power or outputof the light source may be controlled by the imaging system 500 (e.g.,by the controller 530) or may be controlled by a system external to theimaging system 500 (e.g., by an external workstation or processor, suchas a processor of a navigation system).

In some examples, the optical assembly 505, zoom actuator 520, focusactuator 525 and camera 535 may all be housed within a single housing(not shown) of the imaging system. In some examples, the controller 530,memory 550, 3D scanner 545, wireless transceiver, power source and/orlight source may also be housed within the housing.

In some examples, the imaging system 500 may also provide mechanisms toenable manual adjusting of the zoom and/or focus optics 510, 515,similarly to conventional systems. Such manual adjusting may be enabledin addition to motorized adjusting of zoom and focus. In some examples,such manual adjusting may be enabled in response to user selection of a“manual mode” on a user interface.

The imaging system 500 may be mountable on a moveable support structure,such as the positioning system (e.g., robotic arm) of a navigationsystem, a manually operated support arm, a ceiling mounted support, amoveable frame, or other such support structure. The imaging system 500may be removably mounted on the moveable support structure. In someexamples, the imaging system 500 may include a support connector (e.g.,a mechanical coupling) to enable the imaging system 500 to be quicklyand easily mounted or dismounted from the support structure. The supportconnector on the imaging system 500 may be configured to be suitable forconnecting with a typical complementary connector on the supportstructure (e.g., as designed for typical end effectors). In someexamples, the imaging system 500 may be mounted to the support structuretogether with other end effectors, or may be mounted to the supportstructure via another end effector.

When mounted, the imaging system 500 may be at a known fixed positionand orientation relative to the support structure (e.g., by calibratingthe position and orientation of the imaging system 500 after mounting).In this way, by determining the position and orientation of the supportstructure (e.g., using a navigation system or by tracking the movementof the support structure from a known starting point), the position andorientation of the imaging system 500 may also be determined. In someexamples, the imaging system 500 may include a manual release buttonthat, when actuated, enable the imaging system 500 to be manuallypositioned (e.g., without software control by the support structure).

In some examples, where the imaging system 500 is intended to be used ina navigation system environment, the imaging system 500 may include anarray of trackable markers, which may be mounted on a frame on theimaging system 500) to enable the navigation system to track theposition and orientation of the imaging system 500. Alternatively oradditionally, the moveable support structure (e.g., a positioning systemof the navigation system) on which the imaging system 500 is mounted maybe tracked by the navigation system and the position and orientation ofthe imaging system 500 may be determined using the known position andorientation of the imaging system 500 relative to the moveable supportstructure.

The trackable markers may include passive reflective tracking spheres,active infrared (IR) markers, active light emitting diodes (LEDs), agraphical pattern, or a combination thereof. There may be at least threetrackable markers provided on a frame to enable tracking of position andorientation. In some examples, there may be four passive reflectivetracking spheres coupled to the frame. While some specific examples ofthe type and number of trackable markers have been given, any suitabletrackable marker and configuration may be used, as appropriate.

Determination of the position and orientation of the imaging system 500relative to the viewing target may be performed by a processor externalto the imaging system 500 (e.g., a processor of the navigation system).Information about the position and orientation of the imaging system 500may be used, together with a robotic positioning system, to maintainalignment of the imaging system 500 with the viewing target (e.g., toview down an access port during port-based surgery) throughout themedical procedure.

For example, the navigation system may track the position andorientation of the positioning system and/or the imaging system 500either collectively or independently. Using this information as well astracking of the access port, the navigation system may determine thedesired joint positions for the positioning system so as to maneuver theimaging system 500 to the appropriate position and orientation tomaintain alignment with the viewing target (e.g., the longitudinal axesof the imaging system 500 and the access port being aligned). Thisalignment may be maintained throughout the medical procedureautomatically, without requiring explicit control input. In someexamples, the operator may be able to manually move the positioningsystem and/or the imaging system 500 (e.g., after actuation of a manualrelease button). During such manual movement, the navigation system maycontinue to track the position and orientation of the positioning systemand/or the imaging system 500. After completion of manual movement, thenavigation system may (e.g., in response to user input, such as using afoot pedal, indicating that manual movement is complete) reposition andreorient the positioning system and the imaging system 500 to regainalignment with the access port.

The controller 530 may use information about the position andorientation of the imaging system 500 to perform autofocusing. Forexample, the controller 530 may determine the working distance betweenthe imaging system 500 and the viewing target and thus determine thedesired positioning of the focus optics 515 (e.g., using appropriateequations to calculate the appropriate positioning of the focus optics515 to achieve a focused image) and move the focus optics 515, using thefocus actuator 525, in order to bring the image into focus. For example,the position of the viewing target may be determined by a navigationsystem.

The working distance may be determined by the controller 530 usinginformation (e.g., received from the navigation system, from thepositioning system or other external system) about the position andorientation of the imaging system 500 and/or the positioning systemrelative to the viewing target. In some examples, the working distancemay be determined by the controller 530 using an infrared light (notshown) mounted on near the distal end of the imaging system 500.

In some examples, the controller 530 may perform autofocusing withoutinformation about the position and orientation of the imaging system500. For example, the controller 530 may control the focus actuator 525to move the focus optics 515 into a range of focus positions and controlthe camera 535 to capture image data at each focus position. Thecontroller 530 may then perform image processing on the captured imagesto determine which focus position has the sharpest image and determinethis focus position to be the desired position of the focus optics 515.The controller 530 may then control the focus actuator 525 to move thefocus optics 515 to the desired position. Any other autofocus routine,such as those suitable for handheld cameras, may be implemented by thecontroller 530 as appropriate.

In some example, the viewing target may be dynamically defined by thesurgeon (e.g., using a user interface provided by a workstation, bytouching the desired target on a touch-sensitive display, by using eyeor head tracking to detect a point at which the surgeon's gaze isfocused and/or by voice command), and the imaging system 500 may performautofocusing to dynamically focus the image on the defined viewingtarget. This may enable the surgeon to focus an image on differentpoints within a field of view, without changing the field of view andwithout having to manually adjust the focus of the imaging system 500.

In some examples, the imaging system 500 may be configured to performautofocusing relative to an instrument using in the medical procedure.An example of this is shown in FIG. 11. For example, the position andorientation of a medical instrument, such as a tracked pointer 222, maybe determined and the controller 530 may perform autofocusing to focusthe captured image on a point defined relative to the medicalinstrument. In the examples shown in FIG. 11, the tracked pointer 222may have a defined focus point at the distal tip of the pointer 222. Asthe tracked pointer 222 is moved, the working distance between theoptical imaging system 500 and the defined focus point (at the distaltip of the tracked pointer 222) changes (from D1 in the left image to D2in the right image, for example). The autofocusing may be performedsimilarly to that described above, however instead of autofocusing on aviewing target in the surgical field, the imaging system 500 may focuson a focus point that is defined relative to the medical instrument. Themedical instrument may be used in the surgical field to guide theimaging system 500 to autofocus on different points in the surgicalfield, as discussed below. This may enable a surgeon to change the focuswithin a field of view (e.g., focus on a point other than at the centerof the field of view), without changing the field of view and withoutneeding to manually adjust the focus of the imaging system 500. Wherethe field of view includes objects at different depths, the surgeon mayuse the medical instrument (e.g., a pointer) to indicate to the imagingsystem 500 the object and/or depth desired for autofocusing.

For example, the controller 530 may receive information about theposition and orientation of a medical instrument. This position andorientation information may be received from an external source (e.g.,from an external system tracking the medical instrument or from themedical instrument itself) or may be received from another component ofthe imaging system 500 (e.g., an infrared sensor or a machine visioncomponent of the imaging system 500). The controller 530 may determine afocus point relative to the position and orientation of the medicalinstrument. The focus point may be predefined for a given medicalinstrument (e.g., the distal tip of a pointer, the distal end of acatheter, the distal end of an access port, the distal end of a softtissue resector, the distal end of a suction, the target of a laser, orthe distal tip of a scalpel), and may be different for different medicalinstruments. The controller 530 may use this information, together withinformation about the known position and orientation of the imagingsystem 500 (e.g., determined as discussed above) in order to determinethe desired position of the focus optics 515 to achieve an image focusedon the focus point defined relative to the medical instrument.

In examples where the imaging system 500 is used with a navigationsystem 205 (see FIG. 2B), the position and orientation of a medicalinstrument (e.g., a tracked pointer 222 or a tracked port 210) may betracked and determined by the navigation system 205. The controller 530of the imaging system 500 may automatically autofocus the imaging system500 to a predetermined point relative to the tracked medical instrument(e.g., autofocus on the tip of the tracked pointer 222 or on the distalend of the access port 210). Autofocusing may be performed relative toother medical instruments and other tools that may be used in themedical procedure.

In some examples, the imaging system 500 may be configured to performautofocusing relative to a medical instrument only when it is determinedthat the focus point relative to the medical instrument is within thefield of view of the imaging system 500. This may avoid an unintentionalchange of focus when a medical instrument is moved in the vicinity ofbut outside the field of view of the imaging system 500. In exampleswhere the imaging system 500 is mounted on a moveable support system(e.g., a robotic arm), if the focus point of the medical instrument isoutside of the current field of view of the imaging system 500, themoveable support system may position and orient the imaging system 500to bring the focus point of the medical instrument within the field ofview of the imaging system 500, in response to input (e.g., in responseto user command via a user interface or voice input, or via activationof a foot pedal).

The imaging system 500 may be configured to implement a small time lagbefore performing autofocus relative to a medical instrument, in orderto avoid erroneously changing focus while the focus point of the medicalinstrument is brought into and out of the field of view. For example,the imaging system 500 may be configured to autofocus on the focus pointonly after it has been substantially stationary for a predeterminedlength of time (e.g., 0.5 s to 1 s).

In some examples, the imaging system 500 may also be configured toperforming zooming with the focus point as the zoom center. For example,while a focus point is in the field of view or after autofocusing on acertain point in the field of view, the user may provide command input(e.g., via a user interface, voice input or activation of a foot pedal)to instruct the imaging system 500 to zoom in on the focus point. Thecontroller 530 may then position the zoom optics 520 accordingly to zoomin on the focus point. Where appropriate, the positioning system (if theimaging system 500 is mounted on a positioning system) may automaticallyreposition the imaging system 500 as needed to center the zoomed in viewon the focus point.

In some examples, the imaging system 500 may automatically changebetween different autofocus modes. For example, if the current field ofview does not include any focus point defined by a medical instrument,the controller 530 may perform autofocus based on a preset criteria(e.g., to obtain the sharpest image or to focus on the center of thefield of view). When a focus point defined by a medical instrument isbrought into the field of view, the controller 530 may automaticallyswitch mode to autofocus on the focus point. In some examples, theimaging system 500 may change between different autofocus modes inresponse to user input (e.g., in response to user command via a userinterface, voice input, or activation of a foot pedal).

In various examples of autofocusing, whether or not relative to amedical instrument, the imaging system 500 may be configured to maintainthe focus as the zoom is adjusted.

In some examples, the imaging system 500 may generate a depth map. Thismay be performed by capturing images of the same field of view, but withthe imaging system 500 focused on points at different depths to simulate3D depth perception. For example, the imaging system 500 mayautomatically perform autofocusing through a predefined depth range(e.g., through a depth of about 1 cm) and capturing focused images atdifferent depths (e.g., at increments of 1 mm) through the depth range.The images captured at different depths may be transmitted to anexternal system (e.g., an image viewing workstation) where they may beaggregated into a set of depth images to form a depth map for the samefield of view. The depth map may provide focused views of the field ofview, at different depths, and may include contours, color-coding and/orother indicators of different depths. The external system may provide auser interface that allows a user to navigate through the depth map.

In some examples, the optical imaging system 500 could be configuredwith a relatively large depth of field. The 3D scanner 545 may be usedto create a depth map of the viewed area, and the depth map may beregistered to the image captured by the camera 535. Image processing maybe performed (e.g., using the controller 530 or an external processor)to generate a pseudo 3D image, for example by visually encoding (e.g.,using color, artificial blurring or other visual symbols) differentparts of the captured image according to the depth information from the3D scanner 545.

FIGS. 7 and 8 are perspective views of an example embodiment of theimaging system 500. In this example, the imaging system 500 is shownmounted to the positioning system 208 (e.g., a robotic arm) of anavigation system. The imaging system 500 is shown with a housing 555that encloses the zoom and focus optics, the zoom and focus actuators,the camera, the controller and the 3D scanner. The housing is providedwith a frame 560 on which trackable markers may be mounted, to enabletracking by the navigation system. The imaging system 500 communicateswith the navigation system via a cable 565 (shown cutoff). The distalend of the imaging system 500 is provided with light sources 570. Theexample shows four broad spectrum LEDs, however more or less lightsources may be used, of any suitable type. Although the light sources570 are shown provided surrounding the aperture 553 of the imagingsystem 500, in other examples the light source(s) 570 may be locatedelsewhere on the imaging system 500. The distal end of the imagingsystem 500 may also include openings 575 for the cameras of theintegrated 3D scanner. A support connector 580 for mounting the imagingsystem 500 to the positioning system 208 is also shown, as well as theframe 560 for mounting trackable markers.

FIG. 9 is a flowchart illustrating an example method of autofocusingduring a medical procedure. The example method 900 may be performedusing an example optical imaging system, as disclosed herein.

At 905, the position and orientation of the imaging system isdetermined. This may be done by tracking the imaging system, byperforming calibration, or by tracking the positioning system on whichthe imaging system is mounted, for example.

At 910, the working distance between the imaging system and the imagingtarget is determined. For example, the position of the imaging targetmay be determined by a navigation system, and this information may beused together with the position and orientation information of theimaging system to determine the working distance.

At 915, the desired position of the focus optics is determined, in orderto achieve a focused image.

At 920, the focus actuator is controlled (e.g., by a controller of theimaging system) to position the focus optics at the desired position.

A focused image may then be captured, for example using a camera of theoptical imaging system.

FIG. 10 is a flowchart illustrating an example method of autofocusingrelative to a medical instrument during a medical procedure. The examplemethod 1000 may be performing using an example optical imaging system asdisclosed herein. The example method 1000 may be similar to the examplemethod 900.

At 1005, the position and orientation of the imaging system isdetermined. This may be done by tracking the imaging system, byperforming calibration, or by tracking the positioning system on whichthe imaging system is mounted, for example.

At 1010, the position and orientation of the medical instrument isdetermined. This may be done by tracking the medical instrument (e.g.,using a navigation system), by sensing the medical instrument (e.g.,using an infrared or machine vision component of the imaging system), orby any other suitable methods.

At 1015, the focus point is determined relative to the medicalinstrument. Determining the focus point may include looking up presetdefinitions (e.g., stored in a database) of focus points for differentmedical instruments, and calculating the focus point for the particularmedical instrument being used.

At 1020, the working distance between the imaging system and the focuspoint is determined.

At 1025, the desired position of the focus optics is determined, inorder to achieve a focused image.

At 1030, the focus actuator is controlled (e.g., by a controller of theimaging system) to position the focus optics at the desired position.

A focused image may then be captured, for example using a camera of theoptical imaging system.

The example methods 900, 1000 described above may be entirely performedby the controller of the imaging system, or may be partly performed bythe controller and partly performed by an external system. For example,one or more of: determining the position/orientation of the imagingsystem, determining the position/orientation of the imaging target ormedical instrument, determining the working distance, or determining thedesired position of the focus optics may be performed by one or moreexternal systems. The controller of the imaging system may simplyreceive commands, from the external system(s) to position the focusoptics at the desired position, or the controller of the imaging systemmay determine the desired position of the focus optics after receivingthe calculated working distance from the external system(s).

While some embodiments or aspects of the present disclosure may beimplemented in fully functioning computers and computer systems, otherembodiments or aspects may be capable of being distributed as acomputing product in a variety of forms and may be capable of beingapplied regardless of the particular type of machine or computerreadable media used to actually effect the distribution.

At least some aspects disclosed may be embodied, at least in part, insoftware. That is, some disclosed techniques and methods may be carriedout in a computer system or other data processing system in response toits processor, such as a microprocessor, executing sequences ofinstructions contained in a memory, such as ROM, volatile RAM,non-volatile memory, cache or a remote storage device.

A computer readable storage medium may be used to store software anddata which when executed by a data processing system causes the systemto perform various methods or techniques of the present disclosure. Theexecutable software and data may be stored in various places includingfor example ROM, volatile RAM, non-volatile memory and/or cache.Portions of this software and/or data may be stored in any one of thesestorage devices.

Examples of computer-readable storage media may include, but are notlimited to, recordable and non-recordable type media such as volatileand non-volatile memory devices, read only memory (ROM), random accessmemory (RAM), flash memory devices, floppy and other removable disks,magnetic disk storage media, optical storage media (e.g., compact discs(CDs), digital versatile disks (DVDs), etc.), among others. Theinstructions can be embodied in digital and analog communication linksfor electrical, optical, acoustical or other forms of propagatedsignals, such as carrier waves, infrared signals, digital signals, andthe like. The storage medium may be the internet cloud, or a computerreadable storage medium such as a disc.

Furthermore, at least some of the methods described herein may becapable of being distributed in a computer program product comprising acomputer readable medium that bears computer usable instructions forexecution by one or more processors, to perform aspects of the methodsdescribed. The medium may be provided in various forms such as, but notlimited to, one or more diskettes, compact disks, tapes, chips, USBkeys, external hard drives, wire-line transmissions, satellitetransmissions, internet transmissions or downloads, magnetic andelectronic storage media, digital and analog signals, and the like. Thecomputer useable instructions may also be in various forms, includingcompiled and non-compiled code.

At least some of the elements of the systems described herein may beimplemented by software, or a combination of software and hardware.Elements of the system that are implemented via software may be writtenin a high-level procedural language such as object oriented programmingor a scripting language. Accordingly, the program code may be written inC, C++, J++, or any other suitable programming language and may comprisemodules or classes, as is known to those skilled in object orientedprogramming. At least some of the elements of the system that areimplemented via software may be written in assembly language, machinelanguage or firmware as needed. In either case, the program code can bestored on storage media or on a computer readable medium that isreadable by a general or special purpose programmable computing devicehaving a processor, an operating system and the associated hardware andsoftware that is necessary to implement the functionality of at leastone of the embodiments described herein. The program code, when read bythe computing device, configures the computing device to operate in anew, specific and predefined manner in order to perform at least one ofthe methods described herein.

While the teachings described herein are in conjunction with variousembodiments for illustrative purposes, it is not intended that theteachings be limited to such embodiments. On the contrary, the teachingsdescribed and illustrated herein encompass various alternatives,modifications, and equivalents, without departing from the describedembodiments, the general scope of which is defined in the appendedclaims. Except to the extent necessary or inherent in the processesthemselves, no particular order to steps or stages of methods orprocesses described in this disclosure is intended or implied. In manycases the order of process steps may be varied without changing thepurpose, effect, or import of the methods described.

The invention claimed is:
 1. An optical imaging system for imaging atarget during a medical procedure, the system comprising: an opticalassembly comprising moveable zoom optics and moveable focus optics andhaving an aperture; a zoom actuator for positioning the zoom optics; afocus actuator for positioning the focus optics; a controller forcontrolling the zoom actuator and the focus actuator in response toreceived control input; and a camera for capturing an image of thetarget from the optical assembly, wherein the aperture is adjustable,wherein each of the zoom optics and the focus optics comprises aplurality of independently moveable lenses operably coupled with alinear stage and a conveyor system operably coupled with the zoomactuator and the focus actuator for respectively controlling positioningthe zoom optics and the focus optics, the linear stage comprising alinear guide and a guide actuator, and the conveyer system comprising aconveyor belt mechanism, and the conveyor system operably coupled withthe zoom actuator and the focus actuator for respectively controllingpositioning the zoom optics and the focus optics, wherein the zoomoptics and the focus optics are independently moveable by the controllerusing the zoom actuator and the focus actuator, respectively, whereinthe optical imaging system is configured to: operate at a minimumworking distance from the target, the minimum working distance definedbetween the aperture of the optical assembly and the target,automatically autofocus to a predetermined point relative to a medicalinstrument only after a time lag, whereby erroneous focus change isavoided, automatically change between different autofocus modes,autofocus through a predefined depth range, and capture focused imagesat different depths through the predefined depth range, and wherein thecontroller is configured to: determine the minimum working distance byreceiving information from a navigation system external to the opticalimaging system, the information comprising information relating to aposition and an orientation of a positioning system and informationrelating to a position and an orientation of the medical instrument byusing a machine vision component, determine a desired position of thefocus optics based on the minimum working distance, and control thefocus actuator to position the focus optics at the desired position. 2.The optical imaging system of claim 1, wherein the optical imagingsystem is configured to mount in relation to a moveable supportstructure, and wherein the optical imaging system is configured to oneof automatically autofocus to the predetermined point relative to themedical instrument and automatically autofocus to the predeterminedpoint relative to the medical instrument only if a focus point, relativeto the medical instrument, is within a field of view of the opticalimaging system, whereby unintentional focus change is avoided.
 3. Theoptical imaging system of claim 2, wherein the optical imaging systemfurther comprises a support connector configured to removably mount theoptical imaging system in relation to the moveable support structure. 4.The optical imaging system of claim 2, wherein the moveable supportstructure comprises at least one of: a robotic arm, a manually operatedsupport arm, and a moveable support frame.
 5. The optical imaging systemof claim 2, further comprising a manual release button configured toenable manually positioning the optical imaging system.
 6. The opticalimaging system of claim 1, further comprising a single housingconfigured to accommodate at least one of: the optical assembly, thezoom actuator, the focus actuator, and the camera, wherein the housingis further configured to accommodate the controller.
 7. The opticalimaging system of claim 1, wherein the controller is configured tocommunicate with a processor, the controller responsive to control inputreceived from the processor via a user interface.
 8. The optical imagingsystem of claim 1, wherein the controller is configured to communicatewith a processor, the controller responsive to control input receivedfrom the processor via an input system.
 9. The optical imaging system ofclaim 1, further comprising a three-dimensional (3D) camera configuredto capture 3D information of the target, wherein the 3D camera isconfigured to automatically focus through a predefined depth range andautomatically capture a plurality of focused images corresponding to aplurality of depths through the depth range, wherein the imaging systemis configured to generate a depth map based on the plurality of focusedimages corresponding to the plurality of depths through the depth range,wherein the depth map provides focused views of the field of view,corresponding to the plurality of depths through the depth range, andwherein the depth map comprises at least one of contours, color-coding,and any other indicator of each depth of the plurality of depths. 10.The optical imaging system of claim 1, further comprising at least onelinear stage mechanism configured to move at least one of the zoomoptics and focus optics.
 11. The optical imaging system of claim 1,further comprising at least one of: a power source, a power connectorconfigured to couple with the power source, and a light sourceconfigured to couple with the power source.
 12. The optical imagingsystem of claim 1, further comprising an array of trackable markersconfigured to track position and orientation of the optical imagingsystem by the navigation system.
 13. The optical imaging system of claim1, wherein the minimum working distance comprises a range ofapproximately 15 cm to approximately 75 cm.
 14. The optical imagingsystem of claim 1, wherein the information received from the navigationsystem comprises information relating to the position of the opticalassembly relative to a position of the target.
 15. The optical imagingsystem of claim 1, wherein the optical imaging system is supported by apositioning system, wherein the information received from the navigationsystem comprises information relating to the position of the positioningsystem relative to a position of the target, and wherein the minimumworking distance is determined by using a known position of the opticalassembly relative to the position of the positioning system.
 16. Theoptical imaging system of claim 1, wherein the minimum working distanceis determined as a distance between the aperture of the optical assemblyand the target, the target being a focus point defined relative to themedical instrument having a known position and a known orientation, andwherein the information received from the navigation system comprisesinformation relating to the known position and the known orientation ofthe medical instrument.
 17. The optical imaging system of claim 1,further comprising at least one of: a memory coupled with thecontroller, the memory configured to store image data captured by thecamera; and a wireless transceiver configured to transmit data from theoptical imaging system.
 18. A processor for controlling the opticalimaging system of claim 1, wherein the processor is configured to:provide a user interface configured to receive control input, via aninput device coupled with the processor, for independently controllingthe zoom actuator and the focus actuator; transmit control instructionsto the controller of the optical imaging system to adjust zoom and focusin accordance with the control input; and receive image data from thecamera for outputting to an output device coupled with the processor.19. The processor of claim 18, wherein the optical imaging systemcomprises a three-dimensional (3D) camera for capturing 3D informationof the target, wherein the processor is further configured to: generatea 3D image of the target by using the 3D information; and generate andtransmit an augmented reality image to the output device, and whereinthe 3D image of the target is overlaid on a real-time image of thetarget.
 20. A medical system, the medical system comprising: an opticalimaging system, the optical imaging system comprising: an opticalassembly comprising moveable zoom optics and moveable focus optics andhaving an aperture; a zoom actuator configured to position the zoomoptics; a focus actuator configured to position the focus optics; acontroller configured to control the zoom actuator and the focusactuator in response to received control input; and a camera configuredto capture an image of the target from the optical assembly; apositioning system for positioning the optical imaging system; and anavigation system for tracking each of the optical imaging system andthe positioning system relative to the target, wherein the aperture isadjustable, wherein each of the zoom optics and the focus opticscomprises a plurality of independently moveable lenses operably coupledwith a linear stage and a conveyor system operably coupled with the zoomactuator and the focus actuator for respectively controlling positioningthe zoom optics and the focus optics, the linear stage comprising alinear guide and a guide actuator, and the conveyer system comprising aconveyor belt mechanism, and the conveyor system operably coupled withthe zoom actuator and the focus actuator for respectively controllingpositioning the zoom optics and the focus optics, wherein the zoomoptics and the focus optics are independently moveable by the controllerusing the zoom actuator and the focus actuator, respectively, whereinthe zoom optics and the focus optics are independently moveable by thecontroller using the zoom actuator and the focus actuator, respectively,wherein the optical imaging system is configured to: operate at aminimum working distance from the target, the minimum working distancedefined between the aperture of the optical assembly and the target,automatically change between different autofocus modes, automaticallyautofocus to a predetermined point relative to a medical instrument onlyafter a time lag, whereby erroneous focus change is avoided, autofocusthrough a predefined depth range, and capture focused images atdifferent depths through the predefined depth range, and wherein thecontroller is configured to: determine the minimum working distance byreceiving information from a navigation system external to the opticalimaging system, the information comprising information relating to aposition and an orientation of a positioning system and informationrelating to a position and an orientation of the medical instrument byusing a machine vision component, determine a desired position of thefocus optics based on the minimum working distance, and control thefocus actuator to position the focus optics at a desired position.
 21. Amedical system, the medical system comprising: an optical imagingsystem, the optical imaging system comprising: an optical assemblycomprising moveable zoom optics and moveable focus optics and having anaperture; a zoom actuator configured to position the zoom optics; afocus actuator configured to position the focus optics; a controllerconfigured to control the zoom actuator and the focus actuator inresponse to received control input; and a camera configured to capturean image of the target from the optical assembly; a positioning systemfor positioning the optical imaging system; and a navigation system fortracking each of the optical imaging system and the positioning systemrelative to the target, wherein the aperture is adjustable, wherein eachof the zoom optics and the focus optics comprises a plurality ofindependently moveable lenses operably coupled with a linear stage and aconveyor system operably coupled with the zoom actuator and the focusactuator for respectively controlling positioning the zoom optics andthe focus optics, the linear stage comprising a linear guide and a guideactuator, and the conveyer system comprising a conveyor belt mechanism,and the conveyor system operably coupled with the zoom actuator and thefocus actuator for respectively controlling positioning the zoom opticsand the focus optics, wherein the zoom optics and the focus optics areindependently moveable by the controller using the zoom actuator and thefocus actuator, respectively, wherein the zoom optics and the focusoptics are independently moveable by the controller using the zoomactuator and the focus actuator, respectively, wherein the opticalimaging system is configured to: operate at a minimum working distancefrom the target, the minimum working distance defined between theaperture of the optical assembly and the target, automatically autofocusto a predetermined point relative to a medical instrument only after atime lag, whereby erroneous focus change is avoided, automaticallychange between different autofocus modes, autofocus through a predefineddepth range, and capture focused images at different depths through thepredefined depth range, and wherein the controller is configured to:determine the minimum working distance by receiving information from anavigation system external to the optical imaging system, theinformation comprising information relating to a position and anorientation of a positioning system and information relating to aposition and an orientation of the medical instrument by using a machinevision component, determine a desired position of the focus optics basedon the minimum working distance, and control the focus actuator toposition the focus optics at a desired position.